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ABSTRACT 


This article presents a methodological model that can 
be employed to conduct follow-up studies of preservice and inservice 
teacher training programs. The model first defines a training need by 
y identifying "what should be" and "what is" in.the knowledge, 
as and. consequence (reaction to behavior) competencies of 

fainees. The distance between these two poles is an index of the a 
_ program's effectiveness in teacher training. The needs assessment 
model, which can be used for self, summative, or formative 

'. evaluations, is then implemented through the: 1) listing of 

competencies, 2) surveying of inservice teachers for competency 
attainment, 3) ranking of competencies according to the level of 
importance and the*level of attainment, 4%) comparing of-high priority 
‘competencies to the content.of the training program, and 5) revising 
cf the program or the competency. The model can be adapted and 

. extended to suit other institutions and purposes. (CJ) 
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Oo Teaching competencies derived from process- product studies (Borich, 
i 1979; Borich and Kash, 1979; Medley, 1978; Rosenshine, 1975) have contri- 
pals buted in some degree to the-content of inservice.evaluations of teacher training. 
4 i. 5 
= However, because the intent of these studies has been limited to investigating 
Lay teaching behavior vis-a-vis pupil] outcomes, they have been less useful in 


suggesting the methodology with which institutions could conduct inservice 
; evaluations~ of their training programs. This article presents i natiodalad: 
_ ical model that can be emp loyed in conducting follow- aur studies of preservice 
and inservice training, 
Overview of the Model | 4, 

Training institytions ‘are continually in search of ways to improve’ their > 
training programs. " Methods for determining needed areas of improvement. have 
ranged from telephone surveys to full-fledged research studies complete with 
control and exverimental groups. Perhaps, nost used anong these methods has 
been the follow-up questionnaire mailed to recent graduates for the ourpose 

of eliciting their opinions about the braaping they received and thevextent to 
which this ‘training may or may not have prepared them to meet the immediate 


demands of ‘their teaching environment. hile many of cheese” surveys are 


creatively. and exhaustively carried out. pene ultimate utility rests on the 


—— 


- *This model was penal ‘for the Organizatign for Economic Cooperation and 
Deve lopment (OECD) and: represents a portion of the author's final report as 
a member, of OECDExpert's Committee on the’ Pe sak of Inservice Education. 
_and: Tratnind. See atso the Journal of Teacher Education, 1979, Jan.-Feb. XXX, 
Ry 77-86, for a! ‘related portion of the author's report. 
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extent to which ‘they provide direct and unambiguous recommendations ° 
for program improvement. Unfortunately, the excitement and chal 1&nge of 
the survey process can turn to disappointment when evaluators are faced 
with the task of ‘making tinny guolis recommendations from the data they 


receive. 


One approach to conducting a follow-up study is to design the survey 
‘instrument in such a uay as to "lock in" from the outset the tyne and 
quality of the data that will be received. This can be accomplished by 
‘employing a snecitic model for collecting the data which establishes prior 
“gb questionnaire development the precise scheme by which the data will be 
analyzed and interpreted. This can be done by designing the questionnaire 
in such a manner that respondents provide data in a form that can be weighted 
and prioritized so that responses are linked to a-practical decision framework 
far program imorovement. Conceptual izing possible program weaknesses as 
. relative "needs" and utilizing a needs assessment model for data collection 
‘is one means of accomnlishing this. : 
aa A training need can be defined as a discrepancy between an educational 
goal and trainee performance in relation to this goal. The process of 
identifying training needs can be conceptualized as,a discrenancy analysis 


that identifies the two polar positions of "what is" and "what shouTd be." 


Training programs can apply this model by defining "what is" as the 
measured behaviors, skills and competencies of the trainee and "what 
should be" as. the goals or intents of the training program. The discre- 
pancy' (or distance) between these two poles can then be used as an index 
of ‘the effectiveness of the. training program in reaching its intended 


goals and to identify components of the training process that fail to 


| tg 

engender specified behaviors. Discrepancies can be prioritized by a panel 

of tiwtaays or by. statistica techniques that weight the relative importance 
of each goal. statgeant from v ues assigned to them by the respondents. 

_ Prioritized discrepancies, ra ked in descending order, provide the Framework 
for deciding what parts of the program to modify or revise. 


Hoel | ; | & 
Following are the steps involved in implementing the needs assessment 
model. | 
4./ List competencies . .Competency: statements can be derived from the a 
competency implications of teacher: effectiveness studies or from the intents 
and objectives of.teacher trainers, or both. Inservice teachers, trainees 
and program administrators can assist in this task: by supplying competency 
statements derived from an examination of the activities and materials used 
dn the training program or froin a list of program objectives. All competency 
statements are checked againe program activities and materials to insure 
that they scents represent : ‘program objectives. These competency: 
statements are then uate in donstfucting the survey instrument. 
.__ Survey ene teachers . Allora aamole of the trainees who- 
have completed training are asked to rate (a) the relevance of each. compe- 
tency to their current job) function (or havewivel future job function) -and 
(b) their current level of ‘attainment of-each competency. A tynteai 


questionnaire might take the following two-part format. 
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An alternative format for rating competency attainment is to divide 
each competency statement into "knowledge" competence, - performance" 
competence, and "consequence" competence 


These terms can be defined on 
the questionnaire in the following Manner 


Knowledge competence: Ability to accurately recall 


paraphrase, or summarize the procedural mechanics 


of the behavior on a paper and’ pencil test 
Performance. competence: Ability to accurately execute 


the behavior in areal or simulated environment in 
the presence of an. observer 
" Conseauence competence: Ability to elicit learning 


\. .from pupils (as recorded on tests-of affective “and/ 


or cognitive achievement) by using the behavior in 
the classroom, 


These Cay Inehions require the teacher. to make finer. judgments in rating each 
competency and in turn permit a more refined evaluation of the training PAogr Aas 


A questionnaire. iachoctattaa these competency dimensions might take the 
following form: ue - 


: ‘ Ability to 
= Knowledge Ability Produce Punil 
‘Perceived of Mechanics to Perform . Learning. with 
Competency Importance: of Competency Competency Comoetency 
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Each competency then yields for each respondent three discrepancy scores 
which indicate the effectiveness of the training program in producing 

(a) trainee knowledge. (b) trainee performance, and (c) nupil consequences. 


Using these discrepancy seores as @& guide, components of « the training program 


_ can be’ revised to’ Produce increased "knowledge ," "performancé,," and/or 


"pupil impact." The three discrepancy scores above might indicate that 
field experiences for this competency (performance and corisequence) need 
improvement. but classroom instruction (Knowledge) is adeouates 

De Rank Competencies. Competencies are then ranked according to 


ratings obtained on the questionnaire. . For each competency the difference 


_ between perceived impor tance and percniven Tevel of attainnent is calculated 


across the three. dimensions: knowledge, ‘performance, “and consequence. 
"These differences are ieee according to magnitude or relative weight, 
calculated by multiplying the discrapaniey score by the’ ‘average perceived 
importance determined over all respondents. In the above example, if 
the ayErANE perceived importance of competency 1 were 2. 5 the resulting 


knowledge Seen would be 0. 0, the resulting performance discrepancy 


would. be 2. yy and the resulting consequence discrenancy won be. 5.0. Other’ 
Suinstaneted deemed either more or less. important than‘ this competency vara 
have their discrepancies adjusted accord taaty: This weighted ordering takes 
into. account that a smal discrepancy on one:comnetency - may *be of greater. 

- perceived importance ‘than a. large discrevaicy on another competency. 
discrepancies withsthe greatest positive rank difference would have/the 


highest priority for revising the training program. 
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4, Compare High Priority Competencies with the Content of the Training . 
\ Program. High priority competencies determined from the above analysis 


fare compared to the instructional experiences, components, and materials 
that receive high priority in the training curriculum. The instructional 
staff might examine instructional time devoted to the competency, the 
clarity.of the instruction, adequacy of the training materials, and: the number ~ 
of minutes or hours: aMotted to students for practicing the competency in 

order to determine the emphasis that the training program is actually Siactig 
on the competency. When a doupetanisy is highly valued but poorly performed, 
the ee may derive from insufficient rather than ineffective training. 

f Revise Program or ‘Revise Comnetency. Where possible, the emphasis 


. of the training program is modified to match high priority competencies. 
Or, Tf altering. the training program to-emphasize a particular high ‘priority. 
competency is not cost-effective, other training resources (e.g. self- 
paced modules, programmed texts) or other alternatives (e.g., agencies and 
institutions at which the inservice teacher may obtain the needed training) 


are recommended to program graduates. © 


Characteristics of the Model | 


The needs assessment model can be extended and adapted to meet a variety 
of institutional needs. For svaiples the needs assessment instrument could 
be used in conjunction with a similar survey completed by supervisors or © . 
administrators in order‘ to corroborate the subjective responses of the_ 
. teachers. An evaluation of training, for instance, might be based on the 
mean -discrenancy across teachers and supervisors, thereby taking into account 
a second and presumably more objective group of respondents. Or, esac : 


tencies for which the reported level of attainment differs dramatically 
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from supervisor to teacher can be withheld from analysis pending clarifi- 


cation from other data sources, such as the classroom observation of” | 


' teachers. | 


Evaluations employing the. needs assessment model can have multiple 
piirpiees These purposes derive from the nature of the needs data which - 
can be employed with equal effectiveness for making either formative or 
summative judgnents about the training program. Formative data revealing 
the perceived ‘importance of the competencies taught can serve as a check on 

Pe relevance of the training and as a guide to what additional, training 
may. be needed. “Summative data revealing the’ level’ to which trainees 


attatned the competencies compared with trainees from other programs 


_or institutions can serve as an ovéral’ check on the program. The . 


versatility of these data make the needs assessment modet less restrictive 
and more developmental than other approaches to the evaluation of AV aINING 

: The needs assessment model is essentially a self-evaluative spacers 
which. rélies on the judgments of teachers about thelr own performance. The 


assumption underlying the needs model is that the performer (teacher) can 


. best judge his or her own performance and, ‘when explicitly asked to do 50, . 


can make an objective judgment. This assumntion is most. tenable when the 


purpose of the data collection is the evaluation of training and not the 


evaluation of individual teachers. If desired, efforts to make the needs 


assessment model more objective can include supervisor-administrator ratinas 


or limited follow-up visitation. These additions can enhance the credibi- 
lity of the self-reports and provide an additional vantage point from which 
to judge discrepancies between program intents and the performance of 
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: trainees. - f 
An important pgactical characteristic of the needs assessment model 

is the.ease with which it can be implemented. It is sufficiently direct: 
that data analysis and instrument construction: are no more complex than — 
with any type of follow-up survey; yt the amount and interpretability of 
the data it yields is considerably greater than many types of follow-up 
questionnaires. Consequently, it is a mode) that is easily implementable 
by teacher trainers who need immediate feedback on the effectiveness of 
program experiences and materials and who have limited resources. 

Finally, we must note the definition of evaluation implied by the needs 
| assessment model: “determining the congruence between “what should be" and 
. "what is," i.e., between what the teacher should be able to do and what the 

teacher can do. The evaluation is complete when the training program has 
objectively determined the discrepancy between these two poles. This 

definition calls -for the development of goals and objectives (in the form 
of competency statements) and an assessment of whether these goals and 
objectives have been met. This is accomplished by obtaining self-report 
data about both the perceived varie of the training objectives and the 


‘lével to which trainees have attained the objectives. 
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